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ABSTRACT 


A significant anomaly, Fuel Valve Pilot Seal Extrusion, is affecting the Shuttle 
Primary Reaction Control System (PRCS) Thrusters, and has caused 79 to fail. To help 
address this problem, a Shuttle PRCS Thruster Process Evaluation Team (TPET) was 
formed. The White Sands Test Facility (WSTF) and Boeing members of the TPET have 
identified many discrete valve current trace characteristics that are predictive of the 
problem. However, these are difficult and time consuming to identify and trend by 
manual analysis. Based on this exhaustive analysis over months, 22 thrusters previously 
delivered by the Depot were identified as high risk for flight failures. Although these had 
only recently been installed, they had to be removed from Shuttles OV103 and OV104 
for reprocessing, by directive of the Shuttle Project Office. The resulting impact of the 
thruster removal, replacement, and valve replacement was significant (months of work 
and hundreds of thousands of dollars). Much of this could have been saved had the 
proposed Neural Network (NN) tool described in this paper been in place. 

In addition to the significant benefits to the Shuttle indicated above, the 
development and implementation of this type of testing will be the genesis for potential 
Quality improvements across many areas of WSTF test data analysis and will be shared 
with other NASA centers. Future tests can be designed to incorporate engineering 
experience via Artificial Neural Nets (ANN) into depot level acceptance of hardware. 
Additionally, results were shared with a NASA Engineering and Safety Center (NESC) 
Super Problem Response Team (SPRT). There was extensive interest voiced among 
many different personnel from several centers. There are potential spin-offs of this effort 
that can be directly applied to other data acquisition systems as well as vehicle health 
management for current and future flight vehicles. 

The preliminary ANN tool developed during this fellowship for the Component 
Test Facility (CTF) program was designed with the following concepts in mind: 

1) Engineering expertise can be incorporated into software and utilized as a 
consistent diagnostic tool. 

2) The tool will run in parallel with existing test equipment. 

3) The tool will help technicians in the CTF evaluate and categorize hardware. 

4) Data augmentation, storage, and ease of access will enhance the effectiveness 
of the tools diagnostic capability. 

Further development of this tool is warranted based upon the results to date. Risks have 
been minimized with a proof of concept approach and the cost will be less than one 
refurbished valve. Plans have been developed with budgets and schedules to study, 
configure, test, document, and train operators for the final configuration in the CTF. 
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INTRODUCTION 


Solenoids are used to control fluid flow electronically. A solenoid valve is a 
control unit which, when electrically energized or de-energized, either shuts off or allows 
fluid to flow. The actuator takes the form of an electromagnet. When energized, a 
magnetic field builds up which pulls a plunger or pivoted armature against the action of a 
spring. When de-energized, the plunger or pivoted armature is returned to its original 
position by the spring action. 

With a direct-acting solenoid valve, the seat seal is attached to the solenoid core. 
In the de-energized condition, a seat orifice is closed, which opens when the valve is 
energized. With direct-acting valves, the static pressure forces increase with increasing 
orifice diameter which means that the magnetic forces required to overcome the pressure 
forces, become correspondingly larger. Internally piloted solenoid valves are therefore 
employed for switching higher pressures in conjunction with larger orifice sizes; in this 
case, the differential fluid pressure performs the main work in opening and closing the 
valve. Figure 1 illustrates the RCS pilot actuated valve which is the point of focus for 
this report. 



Figure 1: (Left) A cutaway internal view of the pilot-actuated value, (Middle) a diagram 
of the valve indicating the area of interest, (Right) an enlarged view of the pilot valve and 
extruded seal which hinders flow and reduces the pressure differential. 

The deformation and extrusion of the pilot seal will cause an obstruction of fluid 
flow from the upper chamber during operation. Many papers have been written as to the 
cause of this extrusion and a definitive paper is being published. A full explanation as to 
the cause of pilot seal extrusion may be obtained by contacting the NASA colleague of 
this paper. However, if the seal obstructs the flow, then a sufficient pressure differential 
will not be created thereby causing main stage failure. This problem can be characterized 
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by data acquisition of the current trace in the CTF which would then identify the valve 
for refurbishment. 


ARTIFICIAL NEURAL NETWORKS 

An Artificial Neural Network is modeled upon the human brain’s interconnected 
system of neurons. Neural networks imitate the brain’s ability to sort out patterns and 
learn from trial and error, discerning and extracting the relationships that underlie the 
data with which it is presented. Most neural networks are software simulations run on 
conventional computers. Each neuron in the network has one or more inputs and 
produces an output; each input has a weighting factor, which modifies the value entering 
the neuron. The neuron mathematically manipulates the inputs, and outputs the result. 

The neural network is simply neurons joined together, with the output from one neuron 
becoming input to others until the final output is reached. The network learns when 
examples (with known results) are presented to it; the weighting factors are adjusted by 
training algorithms which bring the final output closer to the known result. Neural 
networks are good at providing very fast, very close approximations of the correct 
answer. Although they are not as well suited as conventional computers for performing 
mathematical calculations, neural networks excel at recognizing shapes or patterns, 
learning from experience, or sorting relevant data from irrelevant. Their applications 1 can 
be categorized into classification, recognition and identification, assessment, monitoring 
and control, forecasting and prediction. 
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Figure 2: Depicting the Digital analogy of a single Neuron with that of its Biological 
equivalent. 
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DATA ANALYSIS 


The first data set analyzed consisted of 20 current traces pulled from nominal and 
failed valves. These were typed utilizing a TPET presentation guideline followed in the 
CTF. A new category was developed for those square traces that were very close to 
nominal. Due to limited data, the NN was trained using a statistical approach. 

Boundaries were developed for the data using 2 standard deviations from the mean which 
limited the training set to 1 5 derived current traces given 5 categories. A back 
propagation neural network was built and trained with this initial data set. A cost 
function was developed to flatten the results outside of the normal NN training process. 

The second data set consisted of 1 7 current traces. The NN found anomalies in 
several of the data traces. It identified 1 of the traces as unknown. On closer inspection 
this trace was taken using gas instead of water. Water traces are more consistent and they 
are the only data to be analyzed with this technique. Another anomaly occurred with a 
valve that went from unknown to round to nominal all within the same test day. This 
valve was thrown out since the data was extremely inconsistent. These 4 pieces of data 
were rejected and not included in this data set augmentation. Another new category was 
developed for valves that were rounded but very close to nominal. Figure 3 shows a 
sample analysis of the nominal data traces. The training data for the NN is located in the 
bottom panel. This represents the mean with 2 standard deviation limits. 


Nominal Currant traces 

581 ,716,814,903 .578a.6G9c,8G9(J.6Q9b ,569,703,578b 



Figure 3: (Upper) Nominal current trace, (Middle) Standard deviation, (Bottom) Mean 
and 2 standard deviation boundaries. 
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Figure 4 depicts all categorical bounds and the finalized 18 training sets used in the NN 
analysis which became the network implemented in the Optics Lab and the CTF. There 
is substantial overlap in the boundaries which indicates that manual analysis would tend 
to be error prone for valves that fell between categories. 



Figure 4: (Upper) Upper Boundary current trace, (Middle) Mean, (Bottom) Lower 
Boundary 


STUDIES CONDUCTED 

Once the NN was trained 2 , then it was necessary to determine its predictive 
capabilities in the presence of noise, bias, and scale factor errors. A noise profile was 
developed to include all three of these characteristics. Though not depicted, the noise 
was normally distributed and increased with respect to time to its stated value in the 
figures to follow. Similarly, the biases where increased as a function of time to mimic 
bias conditions in the presence of scale factor errors. 

Figure 5 shows that without any error, the training set and the extended data set 
obtained zero categorical errors when noise was absent. As the noise increased then so 
did the errors. The magnitude of the noise experienced in the CTF is 1/40 of that which 
would cause any significant degradation in performance. However, small negative biases 
or scale factor problems would tend to create unwanted errors. The NN could have been 
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trained to account for these errors but the training time proved to be unnecessary. This 
result justified the preconditioning of the data by removal of biases and the careful 
determination of scale factors. All of which are easily accomplished when the interfacing 
hardware is built and the software are initialized. 




Figure 5: (Top) Categorization error for the training set as a function of Noise, Bias, and 
Scaling Errors, (Bottom) Categorization error for the data set as a function of noise. 

The importance of passing good hardware vs. refurbishing bad hardware prior to 
an In-Flight Failure (IFF) or In-Flight Anomaly (IFA) should be explored in a little more 
detail than indicated above. Let us presume that instead of looking at numbers of errors 
we determine what types of errors occur given noise. Figure 6 shows in the top panel that 
as noise increases nominal hardware starts to fail. However, the bottom panel indicates 
by comparing the top that no failed hardware ever becomes nominal with an increase in 
noise. Also, the other categories only indicate the degree of failure so misdiagnosing a 
valve due to noise would still require action that would prevent an IFF/IFA. Therefore, 
the worst response that could happen with the present design configuration and an 
inordinate amount of noise would be refurbishment of good valves. The primary failure 
mode is not a safety concern but a cost and schedule consideration. This is an acceptable 
mode of failure which can be caught by monitoring failure rates and data traces which are 
already done in the facility as a matter of course. In addition, software can calculate the 
noise levels during any given run and indicate if this has impacted the ability of the NN 
to categorize the valve properly. 
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The studies above indicate that the present design concept would work very well 
in the CTF facility. All data traces were properly characterized given the statistical 
approach to train the network and correctly functioned under abnormal conditions. These 
promising results prompted the decision to attempt an actual test under simulated as well 
as actual CTF conditions. 




Figure 6: (Top) Nominal categorization error for the full data set as a function of noise, 
(Bottom) Nominal categorization error for the nominal data set as a function of noise. 

PROGRAM IMPLEMENTATION AND DEVELOPMENT 

The Artificial Neural Network (ANN) Algorithm was developed using MATLAB 
and the data recorded from the CTF which was reduced using the Znet tool. To develop 
software in the shortest time frame feasible it would be necessary to use as many existing 
components as possible. Therefore, the NN coefficients were transferred from MATLAB 
to a specially designed C++ program to duplicate the MATLAB results. The Znet 
program was written in C and its developer made minor modification to write the real 
time data to a file wherein the program would call the C++ NN program which would 
write the results to another file. Though dissimilar programs, the interface was reduced 
to reading and writing files. 

Finally, the operator interface had to be designed and developed to convey all 
important information that the technician and engineers currently have to analyze by 
hand. The important information is plotted, compared as well as written. To categorize 
the valve as to type, the nominal limits are plotted in the top panel of Figure 7. The 
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actual trace is simultaneously plotted for visual verification of the type which is printed in 
the title. (The bottom panel duplicates the trace as currently output in the strip charts.) 
The title contains the pilot opening time, bias, and noise characteristics to monitor data 
integrity. 


Fuel Valve Water Response Categorization of Type t Extrusion 
IMS£ Response Nominal 
ANN Response ==> Nominal 



Data Reduction oN=>nB09c.txt 

Trigger time *14.93 (msec) 8ias =G.(Xl938 (amps) Noise =O.OCC67'246(amps) 



Figure 7: (Top) Operator interface showing the nominal boundary, a nominal data run 
and indicators for the category, (Bottom) Strip Chart analog of the current data trace with 
bias and pilot opening time. 


OPTICS LAB TEST 

The decision was made to proceed with actual hardware testing based upon 
encouraging research results and available time to structure a test of a mule valve in the 
Optics Lab. Hardware needed to be designed and built while software was 
simultaneously written and developed to accomplish a test which would put the design 
concept through its paces. 

Figure 8 depicts the recorded results that would have been obtained using the 
ANN tool with valve SN 754. The data indicates a failed valve with a flat response. This 
represents the same operator interface which depicts the valve category, pilot opening 
time, bias, and noise characteristics. The purpose of presenting this valve 
characterization is simply to juxtapose with a like categorization of the mule valve that 
was available for our trial test. 
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Fuel Valve Water Response Categoriiahon of Type 1 Extrusion 
1MSE Response =>Not Nominal 
ANN Response ==>Flat 



Data Reduction 0f===>f754b.ui 

Trigger time s*U39 (msec) Bias -=0.0010187 (amps) Noise ^0.00080696(amps) 
Pilot Opening =12.09 (msec) 



Figure 8: (Top) Operator interface showing the nominal boundary, a nominal data run 
and indicators for the category, (Bottom) Strip Chart analog of the current data trace with 
bias and pilot opening time. 

The test setup in the Optics Lab consisted of a Laptop computer with the ANN 
software installed, a National Instruments DAQ pad, an oscilloscope, amplifier, FET 
switch and mule valve. The DAQ pad had been utilized in previous tests on a different 
project which resulted in several damaged I/O ports. This coupled with the power 
conditions in the lab probably were the cause of larger than expected noise levels. The 
mule valve was never an operational piece of equipment but did serve to simulate what 
could be expected in the CTF facility with actual hardware. 

Therefore, when reviewing the mule valve SN 009 results (Figure 9) we see very 
similar tendencies as the previous failed flat response. The differences are due to the test 
setup and equipment utilized to obtain and analyze the data. The system experienced an 
increase of noise by a factor of 25 and still successfully categorized the valve. This is not 
surprising considering the levels that would be required to create categorical errors 
indicated in the Study section. The data in the CTF is much cleaner then our jury rigged 
testing done in the Optics Lab. The trigger pulse was initiated by hand and turned off by 
hand which is evident by the valve on time and the spike which initiates the current trace. 

Characteristics that caused problems were not directly related to the NN software 
but had more to do with how to identify the trigger pulse in noise and where the pilot 
opening time was located in the noise. To compensate for noise effects, a variable 
bandwidth filter was developed and added to the software which feeds data to the ANN 
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algorithm. With this small modification, any noise problem carried into the CTF facility 
with our test equipment would be neutralized. 


Fuel Valve Water Response Categorization of Type 1 Extrusion 
IMSE Response Not Nominal 
ANN Response =>Fia! 



Data Reduction of=*>Tesi1 1 .txi 

Trigger time *19.99 (msec) Stas *-0.06101? (amps) Noise =0.012733(amps) 
Pilot Opening *11 .63 (msec) 



Figure 9: (Top) Operator interface showing a flat response from the Mule Valve in the 
Optics Lab, (Bottom) Strip Chart analog of the current data trace with bias and pilot 
opening time. 


COMPONENT TEST FACILITY (CTF) 

There was sufficient time during the development of the ANN tool to devise a 
demonstration of its utility with hardware and software in the CTF using actual RCS 
hardware. Cables were modified to run this Tool in parallel with the existing depot level 
test equipment. Valves that had been previously failed were identified for inclusion in 
this test process as well as the mule valve used in the Optics Lab Test. 

The first process involved running the mule valve to verify that the cable 
modification did not make any difference to the results of the data. The first three runs 
were taken without any modification to the test equipment. The next 3 runs were taken 
with cable modifications to the test equipment as well as the ANN tool running in 
parallel. Figure 10 shows the results of all 6 runs. The variation from run to run was 
well within normal run to run variations associated with repeating the tests on the same 
valve. Valve opening times varied by less than .25 milliseconds 
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Verification Mule Valve Before and After Cable Modification F0091/2/3 )B091f2/3 



Difference Trace of FQ091/JS • 10091/2/3 



Figure 10: (Top) CTF verification run of mule valve, (Bottom) Difference trace of valve 
current comparing the modified cables. 

The second process involved running the failed valves (SN 256 and 534) 
previously categorized as rounded by manual analysis. Figure 1 1 shows the results of 
this comparison. The test stand digital recording and the ANN tool recording are plotted 
and then differenced. The noise levels are about 10 times the magnitude as seen in the 
original data. This is due to the DAQ pad, as first seen in the Optics Lab test. The neural 
net was built to handle more noise than this. There is a .02 amp bias indicated as well 
from the DAQ pad but the ANN tool compensates by deleting the bias. Very little time 
was allocated for setting the scaling of the DAQ pad so initial results used a value with 
2% error. This is shown by the difference slope increase. When corrected the scaling 
error was essentially zeroed, however the ANN tool was built to handle larger scaling 
errors so there was no need to compensate for this small error. 

Figure 12 shows the final result for valve 256. Just as it had initially failed in the 
CTF facility prior to running the ANN Tool, it is evident that the valve is not nominal. It 
agrees with previous manual assessments that the valve has rounded characteristics which 
identifies the valve as unacceptable and should be scheduled for refurbishment. 
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Figure 11: (Top) CTF 256 Valve trace using test stand hardware, (Middle) 256 Valve trace using parallel 
ANN Tool, (Bottom) Difference trace of current. 


Fuel Valve Water Response Categorization of Type 1 Extrusion 
IMSE Response — >Nol Nominal 
/ANN Response “>Nominal io Round 



Data Reduction aN=>F2562.txt 

Trigger lime =17.27 (msec) Bias “0.027383 (amps) Noise =0.Q055637(3mps) 
Pitot Opening =13.25 (msec) 



Figure 12: (Top) Operator interface showing a rounded response for the 256 Valve, (Bottom) Strip Chart 
analog of the current data trace with bias and pilot opening time. 
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CONCLUSIONS 


It comes as no surprise that engineering expertise can be incorporated into 
software. Neural networks are suitable for pattern recognition which is the task currently 
required for categorization of the PRCS valves. The difference is that algorithmic 
implementations are consistent and non-subjective tools when compared to manual 
analysis. 

The ANN tool is a software algorithm that will run in parallel with existing test 
equipment. It has been shown that there is no loss of data integrity and with minor 
modifications will represent a diagnostic tool that enhances the CTF’s capability of 
typing and storing data. Further development of this tool is warranted based upon the 
results to date. Implementation risks have been minimized, success has been maximized, 
and the cost will be less than one refurbished valve. 

PROPOSED PROJECT CONTINUATION 

A single data trace was used to characterize and categorize the PRCS valves as a 
suitability test. Additional information in the form of pressure, accelerometer, and Hall- 
Effect traces should be incorporated to completely characterize the status of the thruster 
valve. Therefore, existing and historical data traces should be analyzed to enhance the 
accuracy of the results while augmenting the output of the current tool with the main 
valve opening times. Hardware and software should be configured and incorporated into 
the CTF as an assessment tool. Plans to this effect have been developed and presented 
during the final presentation of results to the WSTF management and staff. 

Future studies would be important to optimize the performance of the neural 
network in the proposed ANN tool. These studies would determine the optimal neural 
network type, number of layers and neurons, training algorithm, and category refinement. 
In addition, the RCS valve simulation should be further development, understood and 
documented. This would help in fully understanding the properties of the valve and 
might be included in the ANN tool which could then predict seal extrusion and useful 
valve life before refurbishment. 
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